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Abstract 


The high cost of textbooks is of concern not only to college students but also to society 
as a whole. Open textbooks promise the same educational benefits as traditional 
textbooks; however, their efficacy remains largely untested. We report on one 
community college’s adoption of a collection of open resources across five different 
mathematics classes. During the 2012 fall semester, 2,043 students in five different 
courses used these open access resources. We present a comparison between the 
previous two years in terms of the number of students who withdrew from the courses 
and the number that completed the courses with a C grade or better. Our analysis 
suggests that while there was likely no change in these educational outcomes, students 
who have access to open access materials collectively saved a significant amount of 
money. Students and faculty were surveyed as to their perceptions of these materials 
and the results were generally favorable. 

Keywords: Open educational resources; open textbooks; electronic textbooks; open 
access; sustainability; mathematics education 
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Introduction 


Wiley and Green (2012) state that “College students spend an average of $900 per year 
on textbooks—26 percent of the cost of tuition at a public, four-year university” (p. 83). 
This percentage is sometimes even higher at community colleges where tuition and fees 
are traditionally lower than at universities. Paying for expensive textbooks seems 
particularly questionable in introductory courses, in which the information being 
presented is largely stable and the knowledge is readily available. Textbooks are only 
part of a larger issue, namely that many students struggle to pay for higher education in 
general and textbooks specifically (Allen, 2011). According to Perry (2012), 

The 812% increase in the price of college textbooks since 
1978 makes the run-up in house prices and housing 
bubble (and subsequent crash) in the 2000s seem rather 
inconsequential, and the nine-fold increase in textbook 
prices also dwarfs the increase in the cost of medical 
services over the last three decades. Compared to the 
250% increase in the Consumer Price Index (CPI) over 
the last 34 years, college textbooks have risen more than 
three times the amount of the average increase for all 
goods and services. 

Martin and Lehren (2012) report that in the United States alone, outstanding student 
loans (part of which funds are used for purchasing textbooks) now total over one trillion 
dollars. Although textbooks are not the exclusive reason for this debt, Buczynski (2007) 
argues that particularly for community college students, textbook prices can be a 
significant part of their overall college expenses. 

Hilton and Wiley (2011) suggest that using open textbooks can provide substantial cost 
savings to students without compromising educational efficacy. Open textbooks are 
similar to traditional textbooks in terms of content; however, they are generally 
available for free in digital format, along with low-cost print copies. Such textbooks are 
classified as being a subset of open educational resources (OER). OER have the 
advantage of being freely available online and have Creative Commons licenses that 
(among other things) allow for them to be reused for free in a variety of contexts 
(Bissell, 2009; D’Antoni, 2009; Downes, 2007). Bliss (2013) writes that 

numerous projects have been undertaken to develop 
OER, including the creation of OpenCourseWare at MIT 
and several other universities (see 
http://www.ocwconsortium.org ), education modules like 
those available by Connexions (see http://cnx.org ), 
openly available textbooks such as those offered by CK- 
12 or the Saylor Foundation (see http://ck12.org : 
http://www.savlor.org ), openly available classes (Fini, et 
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al., 2008), and [some versions of] Massively Open 
Online Courses (MOOC) (Mackness, et al., 2010; Fini, 

2009). 

While little research has been done regarding student perceptions of OER, Lindshield 
and Adhikari (2013) report that students enjoy using open textbooks. 

Because this is a relatively recent phenomenon, little research has been undertaken to 
assess how utilizing OER impacts student learning. Hilton and Laman (2012) reported 
on Houston Community College’s adoption of an open textbook for a psychology class. 
During the fall semester of 2011, 690 students used this open textbook. They stated that 
“Compared with students using a traditional text in the spring of 2011, students who 
used the free online textbook scored higher on departmental final examinations, had 
higher grade point averages in the class and had higher retention rates” (p. 265). In the 
context of a university, Feldstein et al. (2012) found that students in business courses 
that utilized OER were more likely to use the learning materials, and found that these 
courses appeared to have lower withdrawal rates and higher grades than those in classes 
without open materials. Similarly, Wiley et al. (2012) found that a high school district in 
Utah, USA that used open textbooks for its science curriculum saw essentially no change 
in their state standardized test scores. In the present study, we add to the body of 
knowledge surrounding the impact of using open educational resources in a community 
college setting by reporting on Scottsdale Community College’s adoption of open course 
materials across five different math courses. 

Context of the Study 

This study took place at Scottsdale Community College (SCC), a community college in 
Arizona, USA with approximately 10,000 students. Males comprise 54% of its student 
body, and 46% are female;76% of its students are Caucasian, 11% Hispanic, 5 % Native 
American, 4% African American, and 4 % Asian/Pacific Islander. Like many who attend 
college in the United States, some students at SCC have difficulty paying for the cost of 
education. Our survey of 966 mathematics students determined that slightly less than 
half of these students (451) used some combination of loans, grants, and tuition waivers 
to pay for the cost of their education. The difficulties of paying for college may prevent 
some students from purchasing textbooks, potentially limiting their educational growth, 
potentially limiting their access to information and knowledge. 

The mathematics faculty at SCC viewed the shift to open materials as one that could 
provide greater access to learning resources at a lower price, without compromising 
student learning. The decision to adopt open materials evolved over a period of several 
years with different instructors in the Mathematics department taking different 
approaches. In the spring of 2012, department members collectively created a cohesive 
strategy for using open educational resources. Previously open materials had been used 
in only a few classes. However, in the fall of 2012 OER was employed throughout five 
different courses at SCC. That semester, all of the materials requested by the five 
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different math courses were available online for free; in some cases, hard copies of 
textbooks were made available for optional purchase at prices ranging from $13.00 to 
$30.00. We next list the courses that utilized open materials and describe the open 
materials that were employed. 

Introductory Algebra Math 09X. 

Three primary OER were employed in this class. First there was a student workbook, 
written by SCC faculty members. Because many of the instructors at SCC do hands-on 
work with groups and white-board work in class, they wanted materials that would 
support this approach. They created a workbook that contains examples (which 
students complete by watching videos), practice problems, and end-of-lesson 
assessments. The SCC faculty published this workbook using a Creative Commons 
license. Second, they used the Internet Mathematics Assessment System, “a web-based 
math assessment tool for delivery and automatic grading of math homework and tests” 
(Lippman, n.d.). This open source software is licensed under the GNU Public License. 
Finally an OER textbook, CK12 Introductory Algebra (Gloag, Gloag, and Kramer), was 
used primarily as an online resource; few instructors required the book, relying instead 
on the video modules that were part of the workbooks. 

Intermediate Algebra Math I2x. 

Materials similar to those used in Math 09X (albeit at a higher level) were available for 
Math I2x. The textbook in this class was CK12 Intermediate Algebra (Gloag, Gloag, 
Kramer, and Landers). 

College Algebra Math 15X. 

As with the Math 09X and Math I2x courses, Math 15X utilized a student workbook 
created by SCC faculty members using a Creative Commons license and the Internet 
Mathematics Assessment System. It also used online lessons created in Softchalk 
(www.softchalk.com ) that introduces the material to students. These lessons include 
text, videos, applets, images, and comprehension questions that the students answer 
and get a score at the end. Unlike the other resources they employed, Softchalk is not 
open source. Finally, a combination of two textbooks, Precalculus by Lippman and 
Rasmuessan and Precalculus by Stitz and Zeager, was used. SCC took advantage of the 
remixability of OER to combine these two open textbooks into one in order to ensure 
that the final version covered all of the competencies required by Maricopa Community 
Colleges, of which SCC is one. This type of remix is more fully described by Hilton, 
Wiley, Stein, and Johnson (2010). In addition, we note that SCC took advantage of the 
CC-license to make small textbook revisions in order to ensure that the content matched 
the local requirements. 
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Trigonometry MAT182. 

This course utilized a portion of the book Precalculus by Lippman and Rasmuessan, 
along with the Internet Mathematics Assessment System. 

PreCalculus MAT187. 

The resources utilized in this class were similar to those employed in MAT15X. SCC’s 
precalculus course is a combination of college algebra and trigonometry; the college 
algebra portion is the same as what is taught in Math 150. However, Softchalk lessons 
were not used in this course. 


Research Questions and Method 


In this study we had three key research questions. 

1. How much money did students save because of the use of open textbooks? 

2. Did the patterns of retention and student success change after OER was 
implemented? 

3. How did students and faculty perceive OER quality, compared to other 
materials? 

In the fall of 2012, 2,043 students taking 65 sections of math classes used the open 
materials. Forty-two different instructors were involved; 643 students were enrolled in 
MAT09X, 764 in MAT12X, 461 in MAT15X, 95 students in MAT182, and 80 students in 
MAT187. Ninety-three of these students were enrolled in distance versions of the 
respective courses. In order to investigate whether these materials influenced (positively 
or negatively) student learning outcomes we examined institutional data reported by 
SCC on withdrawal rates and those completing the course with a C grade or better. 
These data are publicly available. We also surveyed both students and faculty as to their 
experiences with the open materials. These surveys were based on the work done by 
Bliss, et. al (2013), and were administered during class; 966 students completed the 
survey (this number excludes a small number [approximately 3%] of surveys that were 
discarded because the students evidently randomly bubbled in responses as evidenced 
by their out-of-bounds responses). 
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Results 


Textbook Costs 

SCC faculty members who were surveyed as to the costs of the textbooks they typically 
have required in the past reported that the average cost of these textbooks was $125. 
Some courses ranged as high as $220. As stated above all of the materials necessary for 
the five different math courses included in this study were available online for free; in 
some instances hard copies of textbooks were also made available for optional purchase 
at prices ranging from $13.00 to $30.00. If we assume that all 2,043 students would 
have purchased a $125.00 textbook, and instead used the openly licensed, free online 
materials, the resulting savings would be $255,375.00. Clearly not every student would 
have purchased a new textbook, and some students may have opted to purchase a 
printed copy of the open materials, but this figure does indicate the significant cost 
savings to students that can result from a single semester’s adoption of OER. 

Grades and Completion Rates 

Our second research question asked if there were any changes in student success 
patterns or completion rates in the fall of 2012, the semester in which open educational 
resources were used. Student success patterns were measured by the rate at which 
students passed the math class with a C grade or better. The results for the fall 
semesters during years 2010 to 2012 are shown in Table 1 (we only compared fall 
numbers given that the student population in the courses shifts substantially between 
fall and spring semesters). 

Table 1 

Aggregated Data Fall 2010 - Fall 2012: Overall Grades 


Percentage of students earning 
a C grade or better 

Fall 2010 

Fall 2011 

Fall 2012 

MAT 09X 

62% (790) 

67% (704) 

51% (643) 

MAT I2x 

60% (748) 

63% (721) 

62% (764) 

MAT15X 

65% (448) 

64% (388) 

65% (461) 

MAT 182 

56% (106) 

61% (109) 

58% ( 95 ) 

MAT 187 

53 % ( 72 ) 

48% (82) 

55% (80) 


In order to examine whether the distribution of student success differed significantly 
from 2011 to 2012, we conducted a z-test for comparing proportions for each course. 
The results of these z-tests revealed no significant change in student success rates from 
2011 to 2012, with one notable exception: The percentage of student success in Math 
09X declined significantly to 51% in 2012 compared to percentages of 67% and 62% in 
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the prior two years. This result was significant at the c? = .05 level, z = — 5 . 97 ,p < . 001 . 
Possible reasons for this change are included in the discussion section. Otherwise, it 
does not appear that student success rates significantly varied from the same rate in 
2011. 

A similar pattern was observed when examining the data for course completion rates in 
2012 compared to previous years. The 2012 completion rate in Math 09X was 
significantly lower than the completion rate in the previous year, z = —6.11 f p < .001. 
Again, possible explanations for this change will be subsequently discussed. There did 
not appear to be a drop in completion rate in other courses in the year that OER were 
adopted. Table 2 summarizes completion rates for the classes that were the focus of this 
study. 

Table 2 

Aggregated Data Fall 2010 - Fall 2012: Completion Rates 


Completion rates 

Fall 2010 

Fall 2011 

Fall 2012 

MAT 09X 

75% (790) 

79% (704) 

64% (643) 

MAT I2x 

79% (748) 

76% (721) 

75 % (764) 

MAT 15X 

73 % ( 448 ) 

72% (388) 

76% (461) 

MAT 182 

58% (106) 

72% (109) 

72% (95) 

MAT 187 

68% (72) 

62% (82) 

64% (80) 


Student Perceptions of the Materials 

Our third research question asked how students and faculty at SCC perceived the quality 
and content of the new OER. We surveyed students to learn of their perceptions of the 
OER used in their classes. Students were generally positive about the materials. In total, 
83% either strongly agreed or slightly agreed with the statement, “Overall, the materials 
adequately supported the work I did in class.” Figure 1 more specifically illustrates their 
responses to this statement. 
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Overall, the materials adequately 
supported the work I did in class 



No Response Strongly Slightly agree Neutral Slightly Strongly 

agree disagree disagree 


Figure 1. Student responses to the statement, “Overall, the materials adequately 
supported the work I did in class.” 


Similarly, students tended to agree with the statement, “Overall, the materials 
adequately supported the work I did outside of class,” as outlined in Figure 2. 


Overall, the materials adequately 
supported the work I did outside of class 






426 




285 





139 


9 



34 17 

■■■■■ 


No Response Strongly agree Slightly agree Neutral Slightly Strongly 

disagree disagree 


Figure 2. Student responses to the statement, “Overall, the materials adequately 
supported the work I did outside of class.” 


Students also provided an overall endorsement of the open materials as evidenced by 
Figure 3. 
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I would recommend the use of these 
materials to my classmates. 



No Response Strongly agree Slightly agree Neutral Slightly Strongly 

disagree disagree 


Figure 3. Student responses to the statement, “I would recommend the use of 
these materials to my classmates.” 


The student survey also included free-response questions. Their answers to these 
questions were again generally positive. For example, of the 255 responses to the 
question, “What additional comments do you have regarding the quality of the open 
materials used in your class?” 210 (82%) were positive. Representative comments 
include the following statements: “Great!” “They were good. Definitely worth not 
having a massed produced book for.” “I never had an open materials class before. It 
made work less stressful and learning more enjoyable - didn’t constantly feel frustrated 
and was able to look through notes when I got stuck.” “The quality was excellent. It 
really helped my understanding.” 

Several of the answers to this question focused on the price of the materials such as the 
following: “Keep them, buying textbooks is out of date and I think materials should be 
inexpensive.” “I love saving money, I am poor.” “I like the open materials, textbooks are 
so expensive that it makes me not want to buy them.” 

Of the student comments not coded as positive, representative statements included the 
following: “That was fine but you need better material,” or “Fine. I used Youtube a lot to 
get help and other math websites.” 

Faculty Perceptions of the Materials 

Twenty faculty members at SCC completed our survey (response rate of 48%) regarding 
their opinions on the OER that they had employed. In general they viewed the open 
materials positively. In response to the question, “How would you rate the quality of the 
OER textbook used for this course?” three instructors rated the book as “WORSE than 
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the quality of texts in my other courses”; nine instructors said they were “About the 
SAME AS the quality of texts in my other courses”; and six reported that they were 
“BETTER than the quality of texts in my other courses” (two faculty members left this 
question blank). 

Of the 15 instructors who responded to the question, “Do you feel that the OER 
materials adequately supported the work that was completed INSIDE the classroom? 
Why or why not?” 13 answered “yes,” and then provided an explanatory comment. The 
following are two examples of these answers: “Yes, although I will rearrange some of the 
material next time around. For example, I like to cover properties of exponents 
BEFORE exponential and logarithmic functions as we use these properties at that time 
in my class” and “Yes, it is a complete resource.” The two responses that did not begin 
with a “yes” are as follows: “Need more specific examples” and “Most of the time it was 
supported.” 

Similarly, of the 15 instructors who responded to the question, “Do you feel that the 
OER materials adequately supported the work that was completed OUTSIDE the 
classroom? Why or why not?” 13 answered “yes.” Sample responses are “Yes. These 
materials provided the students useful resources” and “Yes. It focused in the lesson 
well.” The remaining two responses were “Mostly - often students had online questions, 
which were discussed through MathAS or in class” and “In Math I2x, the online HW 
contained questions that were not written in the style of the workbook questions. Some 
students had difficulty in transferring their skills to the online questions.” 


Discussion and Limitations 


Our first research question pertained to the amount of money students saved as a result 
of the open textbooks. As we reported above, if all 2,043 students would have purchased 
a $125.00 textbook, and instead used the openly licensed, free online materials, the total 
savings would have been $255,375.00. Because we did not directly measure the amount 
of money students spent purchasing textbooks during the years 2010-2012, we cannot 
ultimately determine what the exact cost savings were. However, even if only half the 
students would have purchased the traditional textbooks, the savings experienced by the 
students at one community college in one semester would still have eclipsed 
$100,000.00. 

To answer our second research question, we examined the rates of student success and 
course completion in the year of OER adoption compared to the previous two years. 
Although this case study prohibits any causal attribution to observed change or lack of 
change in this data, the patterns might provide rationale and direction for future 
research. 
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With the exception of Math 09X, there were no changes observed in either rates of 
success or completion. While there was an observed negative change in both student 
success and completion rates for the Math 09X course, there were external factors that 
suggest that this change may not have been due to the adoption of a new curriculum. 
Immediately prior to the fall 2012 semester, SCC changed the placement system they 
were using to determine which math courses students would take. As a result, students 
who in previous years would have been placed into a lower math course based on their 
placement test scores were instead put into Math 09X. In this case, this policy change is 
perfectly confounded with the adoption of open materials. Given the similarity in the 
process and content of the materials in Math 09X to other courses, and the absence of a 
similar pattern of change in those other courses, there is likely reason to doubt that the 
change in student success and completion for that course is attributable to OER 
adoption. Future studies and evaluation cycles at SCC might seek to further explore 
whether the OER in Math 09X functions differently than the OER in other courses and 
to what extent this policy change at SCC impacts student success and completion in 
Math 09X. 

The patterns of student and faculty responses to the open texts used reveal largely 
favorable receptions to the open resources. Students were not specifically asked to 
compare their experience with the open texts with other traditional curricular materials 
they had used, but their responses indicated an overall level of satisfaction with the 
OER. Perhaps not surprisingly, in open responses, students focused on the cost of the 
books rather than the quality of the books in comparative statements. 

Faculty, on the other hand, were asked to make comparisons between the quality of the 
open and traditional materials. While responses tended to favor open materials in terms 
of overall quality, there was disagreement among the instructors. This is perhaps not 
surprising considering historic diversities of preference of faculty with regard to 
textbooks and a wide range of other learning materials. But one interesting thread 
coming from teacher open responses was a sense that teachers were aware that flaws in 
open textbooks could be readily fixed. For example, in response to the question, “Do you 
feel that the OER materials adequately supported the work that was completed 
OUTSIDE the classroom? Why or why not” one teacher answered, “Yes, although I will 
rearrange some of the material next time around. For example, I like to cover properties 
of exponents BEFORE exponential and logarithmic functions as we use these properties 
at that time in my class.” Otherwise, teachers responded mostly positively about the 
utility of texts in promoting student learning both inside and outside of class. 

It seems worth noting that the ability to ask these kinds of questions to teachers and 
students highlights a unique opportunity for evaluation of open educational resources. 
Because any materials used or produced are readily revised, remixed, reused, and 
redistributed at the campus level, student and faculty perceptions can be easily 
leveraged to provide real, meaningful improvement of materials for subsequent 
iterations of the course. The same questions asked of traditional textbooks, in contrast, 
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would by necessity be purely academic, since copyright restrictions would prohibit 
faculty from making substantive changes to those materials. 

Ultimately, we believe that our key findings are that (l) there was not a strong 
correlation between using an open textbook and changes in student learning and (2) 
students whose faculty assign open textbooks potentially save a significant amount of 
money. OER can be used to drastically improve the affordability of education while 
adhering to the medical ethics maxim primum non nocere, "first, do no harm". 


Conclusion 


This study may represent an important step in examining how the use of open materials 
may influence both the cost of post-secondary education and student learning. While we 
can be certain that students and faculty members viewed these particular materials 
positively, it is not clear whether they had any impact on student learning. Additional 
research is needed to answer important questions in this area of study. For example, a 
future study could determine whether or not the use of open resources has a statistically 
significant effect on students obtaining and using resources for a course. It is possible 
that many students do not get the required and recommended materials for a course 
due to the cost of the materials. Does having all materials for courses freely online 
increase student access to these learning materials? Moreover, does increased access to 
materials lead to increased learning? Do students in distance education perceive OER 
differently from those in a face-to-face setting? Continued work in multiple contexts is 
needed to gain a more detailed view of what happens when OER are substituted for 
traditional learning materials. 

Notice of full disclosure: Some of the authors of this study were involved in the 
adoption and modification of open materials discussed in this article. 

This study was funded in part with a grant from the Bill and Melinda Gates Foundation. 

OER Materials Discussed in the Article 

The OER materials discussed in this paper can be found at 
http://sccmath.wordpress.com . Click the tabs at top to 09X, I2x. This will give links to 
the workbook and all videos. For links to the workbooks and textbooks for all classes, go 
to https://score.scottsdalecc.edu/?q=si 3 

For college algebra https://score.scottsdalecc.edu/?q=nodequeue/i 4 

The Internet Mathematics Assessment System is available at http: //www.imathas.com/ . 
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